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Abstract 

Over  the  last  decade,  considerable  emphasis  has  been 
placed  on  tighter  control  over  the  quality  and  fidelity  of 
test  targets  being  used  by  Major  Defense  Acquisition 
Programs  (MDAPs).  To  this  end,  the  Target 
Verification,  Validation  and  Accreditation  (VV&A) 
process  was  developed.  The  VV&A  process  was 
designed  to  ensure  that  any  target  under  development 
could  trace  its  physical  and  signature  characteristics 
and  kinematic  behavior  to  detailed,  intelligence 
community-blessed  threat  specifications.  As  a target 
was  developed,  specification  tracking  focused  on  how 
well  the  target  matched  the  threat  definitions.  This 
often  affected  the  program’s  ability  to  conduct  an 
effective  test  program,  especially  when  the  very  real 
constraints  of  safety,  range  and  treaty  limitations  were 
taken  into  consideration  as  a program  approached  its 
actual  test  period.  The  Ballistic  Missile  Defense 
Organization  (BMDO),  as  an  organization  charged 
with  orchestrating  various  service-executed  MDAPs, 
followed  this  strict  measure  of  adherence  to  threat 
representation  in  its  management  of  the  test  targets 
program. 

With  the  creation  of  the  Missile  Defense  Agency 
(MDA),  and  its  focus  on  developmental  testing  (DT), 
layered  defense  and  spiral  development,  there  is  no 
longer  a need  to  adhere  to  threat-specific  processes  in 
the  test  and  evaluation  (T&E)  program.  What  has 
emerged  is  an  opportunity  to  focus  Ballistic  Missile 
Defense  (BMD)  testing  and  test  target  requirements 
development,  traceability,  and  ultimate  certification  on 
the  fundamental  T&E  goal  of  performing  valid  and 
suitable  tests,  which  are  optimized  to  fully  characterize 
the  performance  capability  of  the  system,  element,  or 
component  under  test.  Thus,  early  in  the  “capabilities- 
based  acquisition  process,”  the  targets  may  or  may  not 
be  as  stressing  to  the  system  as  observed  threats.  Later 
in  the  design  process,  the  testing  should  be  using 
targets  that  may  be  more  stressing  than  any  observed 
threat — perhaps  even  approaching  target  physical 
limitations.  In  this  “capabilities-based”  environment, 
the  charge  to  the  MDA  Test  and  Assessment  Deputate 
is  to  be  more  efficient  and  more  responsive  to  Program 
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needs.  T&E  management  must  insure  that  test 
programs  are  well  planned,  well  designed,  and  that  the 
proper  resources,  including  targets,  are  available  for 
test  execution. 

To  take  advantage  of  this  opportunity  and  ensure  better 
test  effectiveness,  the  MDA  has  implemented  a more 
robust  Requirements  & Certification  (R&C)  Process 
that  is  aimed  specifically  at  providing  the  various 
MDA/Ballistic  Missile  Defense  System  (BMDS) 
Project  Managers  with  the  forum,  format  and  timely 
support  necessary  to  better  optimize  their  test  programs 
under  the  “capability-based  acquisition”  strategy.  The 
improved  Target  R&C  Process  focuses  target 
development  on  test  objectives  from  the  start.  All 
required  target  characteristics  are  derived  from  test 
objectives;  test  objectives  are  designed  to  capture 
specific  measurable  data  required  to  increase  the 
confidence  with  which  performance  capability  can  be 
described.  Throughout  the  entire  process  of  target 
development,  delivery  and  characterization,  required 
target  characteristics  always  track  back  to  the 
program’s  test  objectives.  The  goal  is  that  all  tests  not 
just  appear  to  turn  out  well  but  that  their  results  are 
meaningful  in  the  context  of  performance  capability 
characterization.  This  new  MDA  Target  R&C  Process 
provides  increased  traceability,  identifies  program 
target  needs  sooner,  and  documents  the  impacts  of  real 
world  constraints  earlier  as  the  target  development  is 
baselined  and  used  for  target  acquisition  control.  As- 
built  targets  are  characterized  and  compared  to  current 
test  objectives.  Suitability  of  the  actual  target  intended 
for  use  in  a specific  test  can  be  confidently  certified.  In 
the  final  analyses,  this  R&C  Process  promises  to 
increase  MDA’s  ability  to  predict  BMD  capability  into 
an  actual  battle  region  with  greater  assurance. 

Overview 

This  paper  addresses  the  target  requirements  and 
certification  process  past  and  present  and  why  change 
is  appropriate  now.  Several  update  objectives  are 
described.  The  old  VV&A  process  (w/  added 
Certification)  is  described  - followed  by  the  new 
integrated  process.  The  performance  of  the  new 
process  against  each  objective  is  reviewed  comparing 
old  and  new.  Also  discussed  is  the  approach  underway 
to  implement  the  updated  process. 

Within  the  MDA,  The  Deputy  for  Test  and  Assessment 
(MDA/TE)  is  charged  with  managing  and  conducting 
T&E  for  the  BMDS  and  its  elements.  Within  the 
Deputate,  the  Director  of  Test  Resources  (MD A/TER) 
is  responsible  for  assuring  that  all  necessary  resources 
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are  available  for  the  conduct  of  BMDS  and  element 
T&E.  Finally,  within  the  Resource  Directorate,  the 
Targets  Requirements  and  Certification  Office 
(MDA/TERF)  is  responsible  for  managing  and 
executing  the  Target  Requirements  and  Certification 
Process  for  all  targets  used  for  BMD  testing.  From  a 
target  perspective,  the  process  begins  with  determining 
what  is  required  to  support  a test  and  concludes  with 
the  determination  that  that  which  is  provided  is  suitable 
to  meet  test  objectives. 


BMDO,  previously  limited  to  operating  as  an 
organization,  focused  on  providing  some  oversight  to 
service  execution  of  BMDS  MDAPs.  Other 
organizations  within  the  Office  of  the  Secretary  of 
Defense  (OSD)  directed  and  controlled  the  target 
VV&A  process.  Fundamentally,  all  targets  were 
required  to  be  threat-based.  All  threats  were  required 
to  be  Defense  Intelligence  Agency-approved. 
Tracking  the  existence  of  test  objectives  was  not  a high 
priority  (if  even  possible). 

Old  VV&A 

The  old  VV&A  process  asked  very  threat-focused 
questions.  Certification  was  later  added  on  to  this 
process  to  address  the  Director’s  concern  that,  even 
with  the  VV&A  process,  threats  and  targets  were  not 
being  adequately  controlled  or  uniformly  applied. 


Within  VV&A,  Verification  (Step  1)  was  often  not 
done  rigorously,  since  existing  threat  definitions  were 
seldom  detailed  enough  to  directly  support  good  test 
engineering.  Where  detailed  threat  definition  was 
available,  it  required  constant  revisiting  and  revision 
due  to  our  understanding  of  the  threat  evolving  over 
time. 


Validation  (Step  2)  looked  only  at  whether  or  not  the 
as-built  target  looked  like  the  latest  threat  definition. 
Validation  did  not  concern  itself  with  what  was 
expected  of  the  target  when  it  was  first  ordered  or  why 
it  might  not  match  either  current  test  requirements  or 
the  threat  as  currently  understood. 


IM  I VSMITFII  , 

VV&A  CHARACTERISTICS 

DEFINED  AND  CONTROLLED  BY  OSD/SEKVICES 
EXECUTED  Ol'TSIDE  BMDO  CONTROL 
FOCUSED  ALMOST  EXCLUSIVELY  ON  THREAT 

MULTIPLE.  OVERLAPPING,  BI  T 
l NCOKREI.ATED  DOCUMENTS  REQUIRED 

VERIFICATION  & ACCREDITA  TION  OFTEN  NO  T 
DONE  - VALIDATION  LOOKED  AT  THREAT 
ONLY 

CERTIFICATION  ADDED  IN  AN  ATTEMPT  TO 
CONTROL  PROCESS  AND  ASSESS  TEST  WORTH 

TEST  IMPACT  NOT  IDENTIFIED  UNTIL  LAST 
MINUTE 


Accreditation  (Step  3)  focused  on  the  MDAPs  view  of 
the  target  and  how  well  it  suited  the  planned  test.  It  did 
not  focus  on  whether  the  target  was  good,  bad,  or 
provided  the  stimuli  needed  for  system 
characterization. 


On  October  12,  2000  the  BMDO  Director  signed  a 
Directive  that  stipulated  that  all  BMD  Testing  would 
be  conducted  with  BMDO  certified  targets.  This 
Directive  also  established  the  Target  Requirements  and 
Certification  Working  Group  (TRCWG)  as  the  body 
that  would  insure  that  target  development  was  traceable 
to  test  planning  and  that  all  targets  were  suitable  to 
meet  test  objectives. 


The  VV&A  process  itself  was  essentially  Operational 
Test  (OT)  driven  and  focused  almost  exclusively  on 
“threat”  realism  (not  on  valid  testing).  Often  threat 
descriptions  continued  to  evolve,  resulting  in  the  never- 
ending  task  of  having  the  target  design  chase  the  threat. 
Meanwhile,  multiple  documents  were  generated  by 
various  participants,  often  containing  similar 
information  from  different  sources  and  each  required 
endless  updating  right  up  until  test  execution. 


Certification  (the  added  step)  was  a BMDO  initiative 
designed  to  focus  management’s  attention  on  the 
quality  and  adequacy  of  targets  being  developed  for 
and  used  by  the  various  MDAP  test  programs.  The 
challenge  was  to  insert  a new  step  into  a well- 
established  process,  and  then  accommodate  a myriad  of 
threat-based  legacy  target  systems  that  were  already 
planned,  paid  for,  and  under  development.  This  step 
was  successful  in  identifying  problems  in  target  design, 
but  occurred  so  late  in  the  target  development  cycle 
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that  there  was  little,  if  any,  time  or  funding  to  initiate 
corrective  action. 

Previous  Process  Flow 

All  recent  attempts  to  rationalize  the  VV&A  (&  C - 
Certification)  process  flow  have  been  forced  to  exist 
within  this  basic,  threat-driven  framework.  Under  the 
BMDO  Directive,  the  process  flow  used  the  TRCWG 
as  the  primary  body  for  attempting  to  impose  order  on 
the  process.  The  initial  target  requirements  (usually 
designed  to  emulate  some  “known”  threat  or  subset 
thereof)  were  developed  by  the  various  element 
program  offices  and  delivered  to  BMDO  in  the  form  of 
a Target  System  Requirements  Document  (TSRD). 
Most  of  these  TSRDs  read  very  much  like  a hardware 
development  specification.  Their  main  intent  was  to 
specify  the  characteristics  of  a target  intended  to 
emulate  some  particular  threat  or  class  of  threat. 
Following  TRCWG  review  of  these  TSRDs,  the 
TRCWG  would  issue  a Target  Execution  Order  (TEO) 
and  submit  both  the  TSRD  and  TEO  to  the  Targets  and 
Countermeasures  Program  Manager  (then  BMDO/TC) 
for  execution. 


The  Target  Program  Office  then  developed  a Target 
Support  Plan  (TSP)  and  submitted  it  to  the  TRCWG  for 
review  to  determine  if  the  target  build  plan  matched  or 
came  close  to  the  target  required.  At  the  same  time,  the 
TSRD  was  still  being  updated  regularly  by  the  element 
program  offices  to  reflect  their  latest  program  changes. 
The  intelligence  community  continued  to  update  its 
understanding  of  the  “real”  threat  even  as  target 
development  was  underway.  Most  recently, 
BMDO/TC  began  the  process  of  developing  a Target 
Program  Baseline  (TPB)  and  a Target  Development 
Plan  (TDP)  and  placing  both  under  configuration 
control  after  a TRCWG  review  to  insure  that  they 
incorporated  all  the  TSRD  requirements. 

The  TRCWG  was  then  to  track  the  target  development 
through  SRR,  PDR,  CDR,  etc.,  verifying  that  any 


changes  from  the  baseline  TSRD  were  agreed  to  by  the 
TRCWG  members  as  still  meeting  program 
requirements.  Any  change  deemed  a major  deviation 
from  the  TSRD  or  approved  baseline  would  need  to  be 
staffed  back  through  the  BMDO  Configuration  Control 
Board  for  approval.  After  the  target  was  built,  the  final 
Target  Validation  Report  (developed  for  and  under 
control  of  the  OSD  or  the  Service  Operational  Test 
Agency)  was  to  be  written.  BMDO/TER  would  then 
generate  another  separate  report  for  certification.  This 
report  addressed  how  the  completed  target  met  the 
TSRD  requirements  (partial  or  total  threat  emulation) 
and  how  the  actual  target  and  deviations  (if  any)  from 
the  original  requirements  would  effect  testing.  The 
BMDO  Director  then  approved  the  Certification 
Report. 

Current  Situation 

The  creation  of  the  MDA  and  the  restructuring  and 
realignment  of  the  various  Service  missile  defense 
programs  created  an  Agency  with  the  authority  and 
responsibility  to  execute  an  integrated  BMDS  program 
and  focus  on  Developmental  Testing.  Individual 
program  elements  are  now  inside  (a  part  of)  MDA. 
The  Director,  MDA  (MDA/D)  has  basic  milestone 
authority  up  to  transition  of  a capability  to  one  or  more 
of  the  Services.  OSD  and  the  Services  no  longer 
control  the  “VV&A”  and  testing  process. 

Previously,  threat-based,  requirements-driven 
acquisition  provided  specific,  point  design  threat 
against  which  a defensive  system  was  designed. 
Targets  attempted  to  match  the  point  design  threat  as 
closely  as  possible.  Now,  under  capability-based 
acquisition,  test  objectives  are  designed  to  explore  the 
full  range  of  the  defensive  system’s  battle  space 
capability  and  are  not  limited  to  demonstrating 
performance  only  against  specific  point  threats. 

Full  performance  capability  characterization  dictates 
targets  be  developed  to  support  rational  test  objectives 
without  regard  for  how  exactly  they  represent  a “real”, 
validated  threat.  Ultimately  any  assessment  of  the 
system  capability  to  address  “actual  threats”  is  done  at 
the  system  engineering  level  when  the  system’s  “Battle 
Space”  capability  is  determined. 

This  performance  characterization  presented  the 
opportunity  to  address  the  shortfalls  in  the  old 
W& A/Certification  process.  MDA  could  now  focus 
test  resources  on  the  conduct  of  the  right  tests  using  the 
right  targets  in  order  to  fully  characterize  BMDS 
performance  capability.  The  process  could  be  made 
both  more  efficient  and  the  products  of  the  process 
could  now  have  more  direct  value  to  the  development 
process. 
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Objectives 

Three  specific  areas  presented  themselves  as  potential 
opportunities  for  improvement: 

> Focus  on  Test  Objectives 

> Increase  Process  Efficiency 

> Become  More  Responsive  to  Programs 
Test  Objective  Focus 

In  an  attempt  to  optimize  target  utility  in  an  era  of 
performance  capability  characterization,  the  driving 
concern  must  be  that  targets  are  suitable  to  support  the 
test  objectives  rather  than  simply  “look  like  the  threat.” 
While  it  is  natural  that  any  target  designed  to  provide 
stimulus  to  a defensive  weapon  system  will  resemble, 
sometimes  even  closely,  an  actual  observed  potential 
adversary  threat,  the  focus  of  the  target  development 
process  should  be  on  assuring  that  the  target  best 
supports  the  testing  being  performed.  An  objective  of 
the  updated  process  is,  therefore,  that  the  required 
target-critical  characteristics  be  matched  to  the  test 
objectives  for  which  they  are  being  provided. 

Process  Efficiency 

By  its  very  nature,  the  process  of  developing  test 
objective-focused  target  requirements,  monitoring  the 
target  development  designed  to  satisfy  these 
requirements,  reporting  on  the  extent  to  which  these 
requirements  have  been  satisfied  and  how  well  they 
continue  to  meet  test  needs,  all  require  review,  analyses 
and  presentation  of  the  same  data.  The  objective  of  the 
updated  process  is,  therefore,  to  streamline  and 
standardize  the  content  and  format  of  the  various 
process  documents. 

Responsiveness 

A major  shortfall  of  the  old  process  was  manifested 
just  before  each  scheduled  test  flight  whenever  a 
potential  problem  with  target  design  was  noted.  By  the 
time  the  issue  was  understood,  there  remained 
essentially  no  time  to  effect  improvements.  A 
corollary  to  this  manifestation  included  having 
incomplete  documentation  of  the  necessary  test 
program  changes  made  along  the  way.  There  are  often 
target  trade-offs  to  be  made  that  are  caused  by  the 
practicality  of  real-world  testing.  For  example,  no 
matter  the  motivation,  it  is  essentially  impossible  to 
actually  test  fly  a “real  threat.”  Range,  safety, 
environmental  factors,  data  needs  and  treaty  constraints 
all  conspire  to  force  the  target  article  and  test  program 
away  from  what  good  engineering  practices  might 
otherwise  suggest.  An  objective  of  the  updated  process 
is,  therefore,  to  document  both  the  necessary  changes 
as  they  are  made  and  to  highlight  any  disconnects  early 


enough  in  the  process  so  constructive  responses  can  be 
determined  and  implemented. 


Updated  Process  Flow 

The  updated  process  can  be  thought  of  as  having  4 
steps: 

> Step  1 - Requirements  Development 
[MDA/TE  lead] 

> Step  2 - Target  Development  Baseline 
[MDA/TC  lead] 

> Step  3 - Target  Build  and  Characterization 
[MDA/TC  lead] 

> Step  4 - Test/Target  Accreditation/Certification 
[MDA/TE  lead] 


i \c»  \smhh> 


TARGET  REQUIREMENTS  /$k 


System,  Element,  Component 
Being  Characterized 
Data  Collection  Needed  to  Support 
Performance  Extrapolation 


''  REQUIRED  AX 
TARGET 
CRITICAL 
CHARACTERISTICS 


t v i w'tKt  n 


Step  1 - Target  Requirements  Development 
Entry  into  the  Requirements  and  Certification  (R&C) 
process  begins  with  determining  the  requirements  for 
the  target.  From  a system  or  element’s  approved 
master  test  plan,  individual  tests  are  designed  to 
address  and  characterize  the  performance  capability  of 
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the  system,  sub-system  or  components.  In  each  of 
these  tests,  specific  test  objectives  are  determined  and, 
in  order  to  satisfy  these  test  objectives,  targets  with 
certain  characteristics  are  required.  These 
requirements  are  documented  in  TSRDs  prepared  by 
MDA  Offices  and  Project  Managers.  The  TRCWG 
reviews  the  TSRDs  to  assure  that  the  required  target 
critical  characteristics  support  the  listed  test  objectives, 
and  MDA/TE  approves  the  TSRDs  for  target 
development.  No  further  updating  is  expected  during 
the  test  program  barring  a wholesale  program  revision. 
MDA/TE  has  the  lead  in  the  entry  step  in  the  updated 
R&C  Process. 

Step  2 - Target  Development  Baseline 
The  next  step  in  the  R&C  Process  takes  the  approved 
target  requirements  from  step  one  and  begins  target 
development.  This  second  step,  under  the  lead  of 
MDA/TC,  is  designed  to  determine  an  acceptable 
target  development  strategy  to  meet  the  program  test 
needs  (cost,  performance,  and  schedule).  Real-world 
issues  (e.g.,  safety,  range,  current  target  design 
availability,  schedule,  cost  and  treaty  constraints,  etc.) 
are  laid  against  the  program  test  requirements  and  a 
TPB  and  a TDP  are  developed.  These  documents  are 
reviewed  by  the  TRCWG  to  assure  continued 
compatibility  between  planned  test  objectives  and 
planned  target  characteristics.  These  characteristics  are 
put  under  configuration  control.  At  this  point,  if  real 
world  constraints  have  dictated  a change  to  executable 
test  objectives,  this  action  is  reviewed,  approved  by 
competent  authority  and  documented  along  with  an 
explanation/  recommendation  for  their  adoption. 

Step  3 - Target  Build  & Characterization 
The  next  step  in  the  R&C  Process  takes  the  baseline 
target  development  plan  from  step  two  and  continues 
target  development.  This  third  step,  also  under  the  lead 
of  MDA/TC,  is  intended  to  design,  develop,  and 
acquire  a suitable  target  to  meet  the  program  test  needs. 
At  each  review  the  TRCWG  continues  to  monitor 
target  development  against  target  requirements. 
Throughout  the  target  development  process  these 
baseline  target  expectations  are  used  to  measure 
progress.  At  the  close  of  this  step  the  actual,  as-built 
target  is  characterized  and  once  again  compared  to  the 
required  target  critical  characteristics.  Full  target 
characterization  is  captured  for  use  by  the  program  in 
its  modeling  and  simulation.  Following  target 
characterization,  MDA/TC  writes  a Target  Validation 
Report  (TVR),  documenting  the  actual  or  predicted 
characteristics  of  the  as-built  target  as  compared  to  the 
required  characteristics  documented  in  the  TDP  along 
with  an  explanation  for  the  differences,  if  any.  The 
TRCWG  reviews  the  TVR  to  assure  continued 
suitability. 


Step  4 - Test/Target  Accreditation  and  Certification 
The  final  step  in  the  R&C  Process  takes  all  of  the 
information  collected  in  the  previous  three  steps  and 
addresses  the  suitability  of  the  as-built  target. 
Although  this  fourth  step  is  under  the  lead  of  MDA/TE, 
the  system/element  project  lead  or  the  Operational 
Testing  Agent  (OTA)  undertakes  the  final  comparative 
analyses  of  the  as-built  target  against  the  final  test 
objectives  planned  for  the  upcoming  test.  MDA/TE 
draffs  and  the  system/element  project  lead  completes 
the  Target  Accreditation  Report  (TAR).  The  TRCWG 
reviews  and  prepares  a Certification  Memorandum  for 
approval  by  the  Director,  MDA.  The  Certification 
package  details  the  extent  and  limitations  of  the 
suitability  of  the  target  to  meet  the  intended  test 
objectives.  Where  necessary,  any  impacts  and/or 

workarounds  are  also  documented. 

Documentation 

At  each  step  in  the  R&C  Process,  each  document  is 
basically  a matrix  reporting  and  tracking  test 
objectives,  target  requirements,  target  design 

parameters,  any  subsequent  changes  in  any  of  these 
baselined  “specs”,  the  rationale  behind  the  changes, 
and  any  resulting  impacts  to  the  test  program.  The 
following  paragraphs  describe  the  required  documents. 
New  documents,  or  updated  documents,  are  noted  with 
the  symbol 

Step  1 - Target  Requirements  Development 
(Verification) 

Target  requirements  are  defined  in  the  Target  System 
Requirements  Document  (TSRD).  As  an  extension  of 
the  MDA/TE-approved  test  plans,  this  first  R&C  step, 
under  the  lead  of  MDA/TE,  is  designed  to  capture  and 
approve  three  basic  test  data  elements: 

♦>  Test  objectives  requiring  specific  target 
characteristics 

❖ Required  target  critical  characteristics  (TCC) 
(with  tolerances) 

❖ Criticality  ranking  of  required  TCC  (to  assist 
in  later  potential  trade-off  analyses) 

Step  2 - Target  Acquisition  Baseline: 

The  Target  Program  Baseline  (TPB)  and  the  Target 
Development  Plan  (TDP)  are  reviewed  by  the 
TRCWG,  placed  under  configuration  control,  and  used 
as  the  baseline  for  both  target  development  and 
program  test  planning.  The  TDP  adds  (❖)  to  the 
Step  1 TSRD  and  provides: 

■ Test  objectives  (♦>  updated  as  required) 
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❖ Required  TCC  (with  tolerances)  (♦!♦  updated 
as  required) 

■ Criticality  ranking  of  TCC  (*t*  updated  as 
required) 

❖ Notes  highlighting  and  explaining  Test 
Objective/Requirements/Criticality  changes 

❖ Expected  target  characteristics  (planned  to 
result  from  target  development/acquisition) 

❖ Notes  highlighting  and  explaining  expected 
target  characteristic  differences  (if  any  are 
anticipated)  from  required  target 
characteristics 
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Step  3 - Target  Characterization/Validation: 

The  comparative  target  validation  analysis  is 
documented  in  a further  update  to  the  previous  matrix 
information.  This  Target  Validation  Report  (TVR)  is 
then  approved  by  the  MDA/TC  (and  in  the  case  of  OT, 
forwarded  to  DOT&E  as  part  of  their  validation  cycle). 
In  any  case,  the  TVR  is  returned  to  the  program  office 
for  use  in  step  four  to  follow.  The  Target  Validation 
Report  (TVR)  adds  (❖)  to  the  Step  2 TDP  and 
provides: 

■ Test  objectives  (♦>  updated  as  required) 


■ Notes  highlighting  and  explaining  expected 
target  characteristic  differences  (if  any)  from 
required  target  characteristics 

❖ Actual  or  predicted,  as-built  target 
characteristics 

❖ Notes  highlighting  and  explaining  actual  or 
predicted,  as-built  target  characteristic 
differences  (if  any)  against  expected  target 
characteristics 

Step  4 Target  Accreditation  and  Certification: 

The  comparative  target  accreditation  analysis  is 
documented  in  a last  update  of  the  previous  matrix 
information.  This  Target  Accreditation  Report  (TAR), 
drafted  by  MDA/TER  and  completed  by  the  PM/OTA, 
is  reviewed  by  the  TRCWG  and  is  forwarded  under  a 
certification  cover  memorandum  by  the  MDA/TE  to 
the  Director,  MDA  for  final  approval  (and  in  the  case 
of  OT,  also  forwarded  to  DOT&E).  This  approval 
constitutes  final  certification  of  this  particular  test 
objective  and  target  combination.  The  TAR,  drafted  by 
MDA/TE  from  the  previous  matrix  information  and 
completed  by  the  PM/OTA,  adds  (*>)  to  the  Step  3 
TVR  and  provides: 

■ Test  objectives  (♦>  updated  as  required  to 
reflect  current  test  plan) 

❖ Required  TCC  (with  tolerances)  (♦>  updated 
as  required) 

■ Criticality  ranking  of  TCC  (♦♦♦  updated  as 
required) 

■ Notes  highlighting  and  explaining  Test 
Objective/Requirements/Criticality  changes 
(*>  adding  further  notes  as  required) 

■ Expected  target  characteristics  (intended  to 
result  from  target  development) 

■ Notes  highlighting  and  explaining  expected 
target  characteristic  differences  (if  any)  from 
expected  target  characteristics 


Required  TCC  (with  tolerances)  (❖  updated 
as  required) 


Actual  or  predicted,  as-built  target 
characteristics 


Criticality  ranking  of  TCC  (❖  updated  as 
required) 

Notes  highlighting  and  explaining  Test 
Objective/Requirements/Criticality  changes 
(♦>  adding  further  notes  as  required) 

Expected  target  characteristics  (intended  to 
result  from  target  development) 


Notes  highlighting  and  explaining  actual 
target  characteristic  differences  (if  any)  from 
required  target  characteristics 

Notes  addressing  the  programmatic 
workarounds  and/or  impacts  of  any 
differences 

Recommendation  as  to  the  extent  and 
limitations  (if  any)  of  target  suitability  to  meet 
as-intended  test  objectives. 
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Review  of  Objectives 

The  remainder  of  this  paper  will  review  the  three 
objectives  established  for  the  R&C  Process 
improvement  exercise.  Does  the  updated  process 

address  the  objectives  for  which  it  was  undertaken? 

Objective  One  - Focus  on  Test  Objectives 
We  asked  first  that  the  target  process  focus  more 
directly  on  test  objectives  rather  than  spend  its  energies 
trying  to  chase  that  day’s  understanding  of  a given 
threat.  Previously,  while  pursuing  an  elusive  and  ever 
evolving  threat  definition,  suitability  of  a target  was 
often  overcome  by  the  need  to  emulate  the  current 
consensus  estimate  of  the  threat.  By  the  time  the  final 
certification  exercise  reviewed  the  test  in  question,  the 
test  and  target  were  no  longer  an  optimum  match  for 
each  other.  However,  by  the  time  this  became  evident, 
little  time  or  resources  remained  with  which  to  address 
the  issue;  and,  with  the  need  to  chase  the  threat  still 
paramount,  the  ability  to  optimize  the  test  program  was 
severely  compromised. 
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Now,  the  new  R&C  Process  has  a test  objective  focus 
from  the  very  beginning.  The  very  definition  of  the 
required  target  is  driven  solely  by  the  objectives  for 
which  the  test  is  being  conducted.  The  only 
requirement  on  the  target  is  that  it  provides  the  proper 
stimulus  so  that  the  tested  system’s  performance 
capability  characterization  can  be  ascertained. 
Traceability  to  threat-related  battle  space  is  still  very 
apparent.  The  target,  however,  can  be  optimized  for 
test  data  collection  rather  than  dubious  threat 
emulation.  Disconnects  between  target  characteristics 
and  test  objective  needs  can  be  identified  early  and 
certification  of  target/test  suitability  follows  directly. 


Objective  Two  - Increase  Efficiency 
We  started  with  a process  that  required  at  least  five 
different  analyses  and  reports  of  findings.  Each 
analysis  started  anew  and  each  report  presented  its 
findings  (or  requirements)  with'  a dissimilar  purpose 
and  format.  When  the  target  requirements  and  VV&A 
processes  were  completed,  a whole  new  certification 
exercise  still  needed  to  be  conducted.  Each  step  along 
the  way  required  duplicating  much  of  the  work  in  the 
previous  steps  and  then  presented  its  results  in  a 
manner  not  readily  comparable  to  the  results  from 
other  steps. 
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Now,  all  steps  in  the  process  have  a common  goal, 
format  and  end  product:  a target  that  is  suitable  to  meet 
intended  test  objectives.  Each  document,  built  in  a 
matrix  fashion,  builds  on  the  contents  of  the  previous 
step.  The  documentation  at  each  step  is  focused  on 
tracking  the  test  objective,  required  target  critical 
characteristics,  and  assuring  the  target,  when  delivered, 
matches  these  requirements.  Status  of  suitability  is 
known  from  the  outset  and  throughout  the  development 
process.  Certification  naturally  follows  with  a 
minimum  of  late-breaking  issues. 

Objective  Three  - Be  More  Responsive 
Previously,  by  the  time  a test  came  around,  traceability 
to  the  original  test  objectives  or  target  requirements 
was  difficult  to  reconstruct.  What  the  target  was 
intended  to  support  and  the  impact  of  any  shortfalls 
was  difficult  to  determine.  This  was  especially 
disruptive  when  test  and  target  disconnects  surfaced 
very  late  in  the  testing  schedule.  It  could  seldom  be 
known  early  on  just  what  certification  could  be 
expected  for  a given  test-target  combination;  and, 
perhaps  worse,  chasing  an  after-the-fact  understanding 
of  suitability  was  clearly  not  getting  optimum  results 
from  the  testing  program. 
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Now,  all  test  objectives  and  required  target  critical 
characteristics  are  documented  and  tracked  right  from 
the  start  of  the  process.  Test  objectives  and  target 
requirements  are  baselined  early  and,  as  development 
proceeds,  shortfalls  can  be  identified  and  addressed 
early.  The  sole  criterion  for  certification  is  the 
suitability  of  the  target  to  meet  test  objectives.  This  is 
explicitly  available  to  all  participants  early  in  the 
process.  Shortfalls  are  more  easily  highlighted  and 
time  is  available  to  affect  a more  optimum  test-target 
match.  The  process  is  therefore  more  responsive  to  the 
needs  of  a dynamic  test  program. 


Summary 

The  development  of  the  BMDS  and  its  elements  has 
transitioned  from  being  requirements-driven  to  being 
capability-driven.  While  overall  system  and  element 
battle  space  must  remain  traceable  to  validated  threats, 
targets  are  not  each  individually  required  to  emulate 
specific  validated  threats.  Rather,  since  testing  is 
designed  to  characterize  the  BMD  system’s/  element’s 
capabilities,  targets  are  required  to  support  obtaining 
credible  results  under  the  test  objectives  for  which  the 
targets  are  being  provided.  Therefore,  the  focus  for 
target  R&C  has  been  refocused  on  providing  test 
objective-relevant  test  articles.  The  sole  criterion  for 
certification  is  that  of  suitability:  that  the  target 
adequately  and  appropriately  satisfies  the  intended  test 
objectives.  Certification  itself  details  the  extent  to 
which  intended  test  objectives  are  credibly  supported 
by  the  target  to  be  used. 

The  updated  process  is  intended  to  focus  the  .target 
R&C  efforts  directly  on  test  objective  support.  It  is 
formatted  to  streamline  documentation  and  to  lessen 
the  duplication  of  effort  required  of  multiple, 
unconnected  documents  by  forcing  a commonality  of 
content  and  format  throughout  the  process.  And, 
finally,  it  is  fashioned  to  provide  insight  and  support  to 
all  facets  of  the  BMD  test  program  early  enough  to  be 
of  use  to  the  participants.  The  goal  of  a more  optimum 
test-target  combination  is  rapidly  becoming  a reality. 
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